Learning a class of large finite state machines with a recurrent neural network

-One o f the issues in any learning model is how it scales with problem size. The problem o f learning finite state machine (FSMs) from examples with recurrent neural networks has been extensively explored. However, these results are somewhat disappointing in the sense that the machines that can be learned are too small to be competitive with existing grammatical inference algorithms. We show that a type o f recurrent neural network (Narendra & Parthasarathy, 1990, IEEE Trans. Neural Networks, 1, 4-27) which has feedback but no hidden state neurons can learn a special type o f F S M called a finite memory machine (FMM) under certain constraints. These machines have a large number o f states (simulations are for 256 and 512 state FMMs) but have minimal order, relatively small depth and little logic when the F M M is implemented as a sequential machine, Keywords---Recurrent neural network, Finite state machine, Grammatical inference, Automata, Sequential machine, Memory, Temporal sequences, NNIIR, NARX.